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PROJECT  OVERVIEW 

The  progress  reported  here  is  part  of  a  long-range  effort  dedicated  to  developing  and 
validating  measures  of  team  cognition,  and  at  the  same  time,  performing  empirical  studies  to 
better  understand  team  cognition  in  the  context  of  military  team  environments.  This  part  of  the 
effort  focuses  on  the  increasingly  common  military  environment  in  which  individuals  who  are 
distributed  in  space  communicate,  share  information,  and  make  critical  decisions  over  a  richly 
interconnected  network.  Warfare  in  this  environment  has  been  termed  “network  centric.” 

The  original  objectives  of  this  project  involved  conducting  three  empirical  studies  to 
examine  the  effects  of  DMEs  (Distributed  Mission  Environments),  in  which  team  members  are 
geographically  dispersed,  on  team  performance,  process,  and  cognition.  In  parallel,  measures  of 
team  cognition  were  further  developed,  validated,  and  extended  to  DMEs.  This  setting  for  this 
research  is  a  synthetic  three-person  team  task  based  on  USAF  Predator  Uninhabited  Air  Vehicle 
operations.  This  synthetic  task  environment  is  housed  in  ASU  East’s  (formerly  NMSU’s) 
CERTT  (Cognitive  Engineering  Research  on  Team  Tasks)  Laboratory.  Not  only  will  results 
from  this  proposed  work  have  implications  for  military  DMEs,  but  they  will  also  extend  the 
scientific  base  of  knowledge  pertaining  to  team  performance,  process,  and  cognition  in  DMEs 
and  the  specific  influence  of  DME  factors  such  as  communication  mode,  familiarity,  and  co¬ 
presence  on  team  cognition. 

This  report  summarizes  progress  during  the  initial  part  of  this  project  carried  out  at  NMSU. 
In  December  2002  the  PI  moved  to  ASU  East  and  the  grant  at  NMSU  was  terminated  at  the  end 
of  its  second  year  (and  a  no  cost  extension  through  May  31, 2003  was  granted  at  NMSU).  The 
report  that  follows  the  final  year  at  ASU  will  provide  a  comprehensive  account  of  the  entire 
three- year  project  at  NMSU  and  then  at  ASU.  Activities  during  this  initial  performance  period 
centered  on  data  collection  and  analysis  associated  with  two  DME  experiments  (data  collection 
for  the  second  experiment  began  in  9/1/02  and  ended  in  12/31/03  and  so  results  are  reported  here 
only  from  the  first  experiment).  Key  findings  from  the  first  experiment  include  1)  limited 
deleterious  effects  of  the  distributed  manipulation  on  team  performance,  2)  significant  effects  of 
workload  on  team  performance,  3)  suggestions  that  team  composition  (i.e.,  gender  mix)  and 
individual  differences  in  working  memory  account  for  significant  team  performance  variance,  4) 
suggestions  that  these  team  and  individual  differences,  as  well  as  the  timing  of  knowledge 
measures  (immediately  after  training  or  the  very  end  of  the  experiment)  may  contribute  to  recent 
lack  of  correlation  between  knowledge  measures  and  team  performance,  and  5)  favorable  results 
in  regard  to  the  measures  of  knowledge  taken  at  the  team  level  (i.e.,  holistic  measures). 

OBJECTIVES 

The  specific  objectives  of  this  project  involved  conducting  three  empirical  studies  to 
examine  the  effects  of  DMEs  (Distributed  Mission  Environments),  in  which  team  members  are 
geographically  dispersed,  on  team  performance,  process,  and  cognition.  In  parallel,  measures  of 
team  cognition  were  further  developed,  validated,  and  extended  to  DMEs. 


STATUS  OF  EFFORT 
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We  were  on  track  toward  accomplishing  our  objectives  in  that  we  have  completed  the  first 
experiment  and  summarize  the  results  below.  Measures  of  team  cognition  were  also  been 
advanced  and  include  the  development  of  new  holistic  measures  of  teamwork  knowledge  and 
situation  awareness  (i.e.,  elicited  the  team  level)  and  the  integration  some  measures  of  individual 
working  memory  capacity  into  our  overall  measurement  paradigm.  We  completed  the  design  of 
the  second  experiment  and  data  collection  associated  with  it  in  late  2002. 


ACCOMPLISHMENTS  AND  NEW  FINDINGS 

Experiment  1:  The  Effect  of  Co-Located  vs.  Distributed  Mission  Environments  on  Team 
Cognition  and  Performance 

Method.  Twenty  3-person  teams  (65%  males,  35%  females)  of  New  Mexico  State 
University  students  voluntarily  participated  in  two  six-hour  sessions  in  exchange  for  $6.00  per 
hour  payment  to  their  organization.  Participants  were  randomly  assigned  to  a  team  and  specific 
role  (AVO,  PLO,  or  DEMPC)  of  the  CERTT  Uninhabited  Air  Vehicle  synthetic  task.  Teams 
were  randomly  assigned  to  either  a  co-located  or  distributed  condition.  In  the  co-located 
condition  team  members  communicated  during  missions  over  headsets,  but  could  see  each  other 
and  other  computer  displays.  Co-located  teams  could  discuss  the  task  face-to-face  between 
missions  and  were  free  to  examine  other  computer  displays  (e.g.,  to  see  what  information  other 
team  members  have  access  to).  In  the  distributed  condition,  the  DEMPC  was  located  in  a 
separate  room  and  the  AVO  and  PLO  were  separated  by  partitions  and  could  never  have  face-to- 
face  contact  or  see  the  displays  of  other  team  members.  All  communication  for  distributed  teams 
occurred  over  headsets. 

A  working  memory  measure  was  administered  prior  to  training.  Then  teams  participated  in 
the  1.5-hour  training  session  (individual  tutorials  and  tests  followed  by  skills  checks)  and  seven 
40-minute  missions  over  the  course  of  the  two  sessions.  The  first  four  missions  were  low 
workload  missions  with  nine  targets  and  the  last  three  were  high  workload  with  20  targets  and 
more  mission  constraints  (hazards,  weather,  etc.).  During  missions  experimenters  observed  team 
process  behaviors  using  an  event-based  measure  and  ratings  of  process  behaviors  and  presented 
situation  awareness  queries  to  participants  individually  and  as  a  team.  Knowledge  measures 
(taskwork,  holistic  taskwork,  teamwork,  holistic  teamwork,)  were  administered  immediately 
after  training  and  after  the  seventh  mission.  Other  measures  were  also  taken  during  the  sessions 
(e.g.  leadership,  social  desirability,  SART,  NASA  TLX),  but  are  not  the  focus  of  this  report. 

Results  and  Discussion.  Data  analyses  on  the  primary  measures  were  carried  out  during  the 
summer  of  2002.  Analysis  on  secondary  measures  is  in  progress.  This  section  highlights  the 
main  findings  relevant  to  the  analysis  of  the  primary  measures.  Although  there  was  a  tendency 
for  co-located  teams  to  have  an  advantage  over  distributed  teams  in  low  workload  missions, 
whereas  distributed  teams  had  an  advantage  in  high  workload  missions,  the  co-located  vs. 
distributed  manipulation  did  not  significantly  affect  team  or  individual  performance  (see  Figure 
1).  However,  performance  was  affected  by  workload  (Mean  team  performance  scores  =  667  for 
low  and  207  for  high;  F(l,18)  =  608.78 ,p<  .01),  with  poorer  performance  in  high  workload 
regardless  of  condition  and  with  DEMPCs,  and  to  a  lesser  extent,  PLO’s,  being  the  roles  most 
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affected  by  an  increase  in  workload.  For  a  number  of  reasons  we  believe  that  the  distributed 
condition  does  have  a  deleterious  effect  on  team  and  individual  performance  compared  to  the  co¬ 
located  condition,  although  the  relatively  subtle  effects  of  this  manipulation  may  have  been 
masked  by  low  statistical  power  combined  with  high  variance  due  to  individual  and  differences 
that  are  described  in  what  follows.  This  hypothesis  has  motivated  the  next  experiment  in  which 
we  will  better  control  for  individual  and  team  variation. 


Figure  1.  Team  performance  on  UAV  task  across  missions  for  10  co-located  and  10 
distributed  teams. 

There  are  a  number  of  reasons  that  lead  us  to  believe  that  there  is  a  distributed 
disadvantage.  First,  various  trends  in  the  performance  data  are  suggestive  of  this  effect.  For 
instance,  on  the  most  critical  component  of  the  performance  score,  number  of  missed  photos,  co¬ 
located  teams  consistently,  though  not  significantly,  miss  fewer  photos  than  distributed  teams 
(See  Figure  2).  Furthermore,  team  process  behavior,  measured  by  proportion  of  appropriate 
behaviors  at  critical  mission  junctures  is  significantly  better  for  co-located  teams  (M=  .63)  than 
for  distributed  teams  (M=  .48;  F  (1, 18)  =  17.30,  p  <  .01).  Also,  holistic  teamwork  knowledge 
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Figure  2.  Missed  photo  penalty  points  for  co-located  and  distributed  teams. 

measured  at  Session  1  was  more  accurate  for  co-located  teams  (M=  27.6)  than  distributed  teams 
M=  25.8;  F  (1,  18)  =  3.9,  p  =  .06)  and  several  knowledge  measures  predicted  performance 
differences  for  co-located  teams,  but  not  distributed  teams  (e.g.,  teamwork  knowledge  accuracy). 
Note  that  while  there  were  other  measures  not  affected  by  the  location  manipulation,  there  were 
no  measures  that  favored  the  distributed  condition.  Although  there  were  no  performance 
differences  in  this  study  due  to  team  member  location,  the  fact  that  team  process  and  knowledge 
were  affected  by  location  and  have  been  positively  correlated  with  team  performance  in  previous 
studies,  lends  support  to  the  proposed  distributed  disadvantage. 

As  mentioned  previously,  our  relatively  low  power,  coupled  with  variance  due  to 
individual  and  team  composition  differences,  may  have  masked  other  interesting  effects  in  this 
setting.  To  illustrate  we  have  rank  ordered  the  teams  in  terms  of  team  performance  averaged 
across  the  seven  missions  (See  Table  1).  Note  that  co-located  teams  either  perform  very  well  or 
very  poorly,  while  distributed  teams  tend  to  cluster  in  the  center  of  the  distribution. 

Questions  about  the  low-scoring  co-located  teams  led  us  to  explore  some  of  the  individual 
and  team  differences  data  more  fully.  It  turns  out  that  some  variance  in  team  performance  is  due 
to  gender  composition  of  teams  with  mixed-gender  teams  performing  more  poorly  (M=  444) 
than  same  gender  teams  (M=  529).  A  Chi  Square  test  of  mixed  vs.  same  gender  by  high  vs.  low 
scoring  teams  indicated  that  this  difference  is  statistically  significant  (A*(l)  =3 .81,/?  =  .05). 

In  addition,  working  memory  capacity  seems  to  account  for  additional  team  performance 
variance.  The  working  memory  task  that  was  used  in  our  study  consisted  of  32  items.  Each  item 
presented  the  participant  with  four  to  seven  words  and  required  them  to  remember  the  last  three 
words  in  order.  The  working  memory  task  yielded  a  separate  score  for  each  member  of  the  team 
and  was  administered  on  an  individual  basis  before  the  team  task  began.  The  importance  of  the 
working  memory  task  was  recently  highlighted  by  the  fairly  large  correlation  (r  (17)  —  .45  ,p 
=  06)  that  was  found  between  a  component  of  the  DEMPC’s  working  memory  score  and  team 
performance  in  high  workload  missions. 

If  teams  are  categorized  on  the  basis  of  working  memory  scores  and  gender  composition, 
we  see  that  Teams  3, 13,  and  14  are  the  only  co-located  teams  that  have  both  mixed  gender 
composition  and  a  low  working  memory  team  score  (i.e.,  below  a  median  cutoff;  see  Table  1). 
Performance  across  all  seven  missions  is  plotted  in  Figure  3  for  the  distributed  teams  and  these 
two  groups  of  co-located  teams.  In  other  words,  these  co-located  teams  lacked  both  the  gender 
composition  and  working  memory  capacity  associated  with  high  performing  co-located  teams. 
When  these  three  teams  are  removed  from  the  analysis,  the  co-located  team  performance  mean 
across  all  missions  is  519  compared  to  467  for  the  distributed  teams.  Whereas  this  overall 
difference  is  only  marginally  significant  (t  (15)  =  1.65,p  =  .12),  the  low  workload  team 
performance  difference  of  741  for  remaining  co-located  teams  and  657  for  distributed  teams  is 
significant  (t  (15)  =  236,  p  =  .03).  The  difference  for  high  workload  missions  (co-located  M= 
221,  distributed  M  =213)  is  not  significant. 
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Table  1.  Teams  ranked  in  order  (lowest  to  highest)  of  team  performance  score.  (Team  20 
was  excluded  due  to  missing  data.) 
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Figure  3.  Team  performance  for  distributed  teams,  three  co-located  teams  (mixed  and  low 
working  memory),  and  remaining  co-located  teams. 
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So  individual  and  team  differences  not  only  seem  to  play  a  major  role  in  this  task,  but  it 
appears  that  by  controlling  for  or  statistically  co-varying  such  differences,  some  of  the  more 
subtle  effects  due  to  environmental  or  training  manipulations  previously  masked  by  such 
variation  may  be  highlighted.  These  differences  may  have  been  more  pronounced  in  this  study 
and  the  previous  one,  due  to  the  fact  that  the  Air  Force  ROTC  participant  pool  used  in  the  first 
study  had  been  depleted  and  so  teams  were  composed  of  members  from  different  organizations 
including  Army  ROTC,  rugby  club,  criminal  justice  organization,  psychology  club,  etc.  We  plan 
to  explore  these  individual  and  team  differences  in  the  proposed  effort  as  well  as  in  the  study 
planned  for  the  fall  of 2002.  In  this  study,  an  additional  cognitive  individual  difference  measure 
will  be  piloted  and  teams  will  be  more  homogeneously  composed. 

The  pattern  of  results  associated  with  the  knowledge  measures  is  also  worthy  of  mention. 
Similar  to  our  previous  study  on  knowledge  sharing,  the  degree  to  which  knowledge  measures 
were  predictive  of  performance  was  weak  at  best.  In  some  cases  (e.g.,  taskwork  role  knowledge 
of  Session  2)  the  correlations  with  performance  were  negative  (r  ( 18)  =  -.67,/K.10).  Further,  in 
this  study,  the  manipulation  of  location  had  little  effect  on  knowledge.  Note  that  situation 
awareness  measures  taken  at  each  mission  and  the  holistic  measures  of  knowledge  fared  better 
than  individual  teamwork  and  taskwork  knowledge  measures  on  these  grounds.  In  our  first 
CERTT  Lab  study,  however,  knowledge  was  more  predictive  of  performance  than  in  recent 
studies.  One  possible  difference  between  Study  1  and  recent  studies  that  could  help  to  explain 
this  lack  of  correlation  is  the  timing  of  the  knowledge  sessions.  In  Study  1  knowledge  was  first 
measured  after  Mission  1 ,  whereas  in  recent  studies  it  was  measured  after  training  and  before 
Mission  1.  Further  in  both  recent  studies  later  knowledge  measures  were  taken  after  the  last 
mission  as  opposed  to  before  the  last  mission  in  Study  1.  Thus,  timing  of  the  knowledge  session 
(either  too  early  for  learning  or  too  late  for  motivated  responses)  may  have  contributed  to  the 
poor  performance  of  the  knowledge  measures  in  the  last  two  studies.  In  addition,  the  individual 
and  team  variance  described  previously  may  also  contribute  to  this  outcome.  In  particular,  the 
negative  correlation  between  taskwork  role  knowledge  and  performance  seems  to  be  exacerbated 
by  the  low  scoring  co-located  teams. 

Another  noteworthy  pattern  related  to  the  knowledge  measures  is  based  on  testing  for  the 
additive  effects  of  team  process  and  holistic  knowledge  on  team  performance  imputed  by  the 
framework  for  team  cognition  (see  Figure  4).  First,  hierarchical  multiple  linear  regressions  were 
run  controlling  first  for  collective  team  knowledge,  and  then  for  team  process.  Each  model  was 
based  on  three  measures  of  team  knowledge  at  both  the  collective  (measured  individually  and 
then  aggregated)  and  holistic  (elicited  at  the  team  level)  levels:  taskwork  knowledge,  teamwork 
knowledge,  and  situation  awareness.  Models  were  obtained  separately  for  the  first  set  of 
taskwork  and  teamwork  measures  and  the  second  set,  with  the  first  set  obtained  after  training  and 
the  second  set  obtained  after  all  seven  missions  (4  low  workload  and  3  high  workload)  had  been 
completed.  The  critical  incident  process  and  situation  awareness  measures  were  averaged  over 
the  20  teams  separately  for  high  or  low  workload,  and  co-located  or  distributed.  Asymptotic 
performance  in  low  workload  (Mission  4)  was  used  as  the  performance  score  for  the  low 
workload  models.  For  high  workload,  performance  scores  from  missions  5-7  were  averaged. 
Each  of  these  measurements  was  taken  for  ten  teams  in  each  condition.  Finally,  each  condition 
was  modeled  for  both  knowledge  sessions.  These  results  appear  in  Table  2. 
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Figure  4. 
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Framework  for  understanding  team  cognition. 


While  these  results  are  admittedly  based  on  rough  estimates  and  small  samples,  some 
interesting  patterns  emerge.  Knowledge  measurements  taken  prior  to  any  missions  show  a 
different  pattern  of  performance  variance  accounted  for  across  conditions  than  do  knowledge 
measures  taken  after  all  missions.  First,  this  finding  lends  support  to  the  conclusion  that  session 
timing  is  critical.  More  specifically,  performance  variance  attributable  to  collective  knowledge 
increases  for  co-located  teams  between  the  two  knowledge  sessions.  Distributed  teams  show  no 
such  change,  while  accounting  for  just  as  much  overall  performance  variance  via  the  later 
influence  of  holistic  knowledge  which  accounts  for  performance  variance  orthogonal  to  the 
variance  accounted  for  by  collective  knowledge  measures  and  critical  incident  process  alone. 

The  differential  impact  of  collective  knowledge  on  team  performance  for  co-located  and 
distributed  teams  after  all  missions  have  been  completed  may  be  suggestive  of  the  differential 
formation  of  team  knowledge  structure,  depending  on  whether  the  team  is  co-located  or 
distributed.  The  collective  measures  were  aggregated  using  the  arithmetic  mean  of  team 
member’s  knowledge  accuracy  scores,  while  the  holistic  scores  are  some  function  of  individual 
team  member  knowledge  and  the  exchange  of  that  knowledge  with  other  team  members.  It  has 
been  suggested  that  the  arithmetic  mean  is  an  appropriate  team-level  aggregation  method  when 
the  individual  scores  can  also  be  combined  additively  (Barrick,  Stewart,  Nuebert,  &  Mount, 
1998).  Thus,  output  in  an  additive  task  environment  might  best  be  described  via  a  team  process 
in  which  members  have  relatively  homogeneous  input  knowledge,  while  the  inadequacy  of  such 
a  process  in  describing  team  output  may  suggest  other,  more  heterogeneous  knowledge 
structures;  e.g.,  those  found  in  compensatory,  conjunctive,  or  disjunctive  task  environments 
(Steiner,  1972),  whose  processes  may  best  be  described  via  input  variance,  input  minimum,  and 
input  maximum,  respectively.  The  results  found  here  using  knowledge  measured  after  all 
missions  might  imply  that  co-located  teams  operated  in  a  more  additive  task  environment  than 
did  distributed  teams,  and  presumably,  by  the  end  of  all  their  missions,  had  a  more  homogeneous 
team  knowledge  structure  than  did  distributed  teams. 


Cooke  -  NMSU  Final 


9 


Table  2.  Change  in  proportion  of  variance  accounted  for  in  team  performance  from 
hierarchical  multiple  linear  regression  models  from  the  framework  for  team  cognition. 


Collective  Knowledge  AR2  =  .549  (+)  AR2  =  .318(-) 

Critical  Incident  Process  AR2  =  .000  (-)  AR2  =  .087  (-) 

Holistic  Knowledge  AR2  =  .152  (-)  AR2  =  .487  (+) 

Total  R2  =  .70 1  Total  R2  =  .892 

Adj.  R2  =  -1.39  Adj.  R2  =  .135 


Co-located-High  Distributed-High 

Collective  Knowledge  AR2  =  .772  (+)**  AR2  =  .293  (-) 

Critical  Incident  Process  AR2  =  .158  (+)**  AR2  =  .000(-) 

Holistic  Knowledge  AR2  =  .056  (+)  AR2  =  .633  (+) 

Total  R2  =  .986  Total  R2  =  .926 
Adj .  R2  =  .890  Adj .  R2  =  .409 


*p<.10  **p<.05;  N  =  10;  +  or  -  indicates  the  measures  influence  on  adjusted  R 

Note:  for  both  collective  and  holistic,  knowledge  is  comprised  of  taskwork  knowledge,  teamwork  knowledge,  and 
situation  awareness. 
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Experiment  2:  The  Effect  of  Co-Located  vs.  Distributed  Mission  Environments  on  Team 
Cognition  and  Performance  Controlling  for  Team  Composition 

In  this  study  we  repeat  the  procedures  of  Experiment  1,  experimentally  controlling  for 
gender  composition  of  teams  (no  mixed  gender  teams)  and  statistically  controlling  for  working 
memory  differences.  In  addition,  only  five  missions,  the  fifth  a  high  workload  mission,  will  be 
completed  and  team  knowledge  will  be  measured  only  once  -  after  Mission  3  (see  Table  3).  By 
removing  some  of  the  sources  of  variance  in  the  previous  study  we  hope  to  get  a  clearer  picture 
of  any  effects  of  distributed  vs.  co-located  mission  environments  on  team  cognition  and 
performance. 

Table  3.  Protocol  for  fall  2002  experiment. 


Setup 

30  min 

Consent 

15  min 

Working  Memory  & 

Processing  Speed  Measures  40  min 

Training  Tutorial 

45  min 

Skills  Training30  min 
Break 

10  min 

Mission  1 

40  min 

Mission  2 

40  min 

Break 

10  min 

Mission  3 

40  min 

Knowledge  Session 

30  min 

Break 

10  min 

Mission  4 

40  min 

Mission  5 

40  min 

Debrief 

10  min 

Backup  data 

5  min 

OTHER  PROGRESS  IN  THIS  PERIOD 

•  Completion  of  upgraded  experimenter  workstation  (better  data  recording 
capabilities  and  better  participant  monitoring  capabilities,  particularly  in  the 
distributed  condition)  and  a  remote  participant  workstation. 

•  Initiation  of  plans  for  a  not-for-profit,  independent  research  institute  that  extends 
work  with  AFOSR  in  CERTT  Lab  to  include  other  studies  of  distributed 
sociotechnical  systems.  This  will  be  located  in  Mesa.  AZ.  Dr.  Cooke  has 
accepted  a  tenured  full  professor  position  at  Arizona  State  University,  East  to 
begin  in  January  2003. 
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•  US  Positioning  (CERTT  Lab  developer)  demonstrated  Intemet2  connectivity 
between  the  CERTT  Lab’s  UAV  tasks  and  Brooks  AFB  C3STARS  lab  in  a  recent 
distributed  simulation. 

•  The  CERTT  Lab  and  research  focusing  on  UAV  command-and-control  will  be 
the  topic  of  a  lead  article  in  an  upcoming  Unmanned  Vehicles. 

IMPLICATIONS 

•  CERTT  facility  improvements  will  facilitate  experimenter  monitoring,  data 

1  collection  and  analysis  and  move  toward  collaborative  and  distributed 

simulations.  These  improvements  have  made  the  experiments  associated  with  this 
effort  possible  and  will  enable  us  to  more  directly  address  Air  Force  critical 
questions  about  distributed  mission  environments  and  training. 

•  Distributed  mission  environments,  while  affecting  team  process  behavior,  and 
team  knowledge  in  negative  ways  and  potentially  stifling  cognitive  homogeneity 
of  team  members,  have  little  affect  on  team  performance.  This  preliminary  result 
has  positive  implications  for  the  effectiveness  of  distributed  environments  in 
military  and  civilian  applications  (e.g.,  distance  education).  However,  this  result 
should  be  interpreted  with  caution  until  additional  data  have  been  collected. 

•  On  the  other  hand,  recent  research  has  demonstrated  the  significant  impact  of 
individual  differences  in  working  memory  and  team  composition  differences  (i.e., 
gender)  on  team  performance.  We  plan  to  direct  our  work  toward  the 
investigation  of  these  factors,  particularly  those  relevant  to  individual  and  team 
cognition.  The  ability  to  account  for  significant  variance  in  team  performance  has 
implications  for  training,  selection,  team  composition,  and  design  interventions 
that  can  improve  that  performance. 

•  The  UAV  ground  control  task  involves  significant  team  cognition  in  terms  of 
background  knowledge,  information  sharing,  and  team  situation  awareness.  Our 
holistic  measures  taken  at  the  team  level  seem  to  do  a  good  job  at  representing 
team  knowledge  and  thus  have  potential  for  performance  prediction.  Performance 
prediction  is  necessary  for  assessment  and  eventual  diagnosis  of  team 
performance.  Measures  of  team  knowledge  extend  outcome  data  and  help  move 
from  assessment  of  performance  toward  its  diagnosis. 
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♦Cooke,  N.  J.,  &  Shope,  S.  M.  (2001).  The  CERTT-UAV  Synthetic  Task: 

Validity,  Flexibility,  Availability.  Paper  presented  at  the  Air  Force  Office  of  Scientific 
Research  Workshop  on  Team  Performance,  October  16-17,  Fairfax,  VA. 
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Consultative  and  Advisory  Functions 
AFRL,  Brooks  AFB 

In  November  of  2002  we  (CERTT  Lab  and  US  Positioning)  participated  in  an  Intemet2 
demonstration  with  AFRL  at  Brooks  AFB  (Sam  Schifflet,  Phil  Tessier)  and  Veridian 
(Charlie  Dean).  The  CERTT  Lab’s  UAV  task  was  connected  over  the  internet  with  the 
C3STARS  AW  ACS  task  at  Brooks.  The  demonstration  was  successful. 

AFRL,  Mesa,  AZ 

In  January  2002  Nancy  Cooke  and  Steven  Shope  presented  their  vision  of  an  independent 
research  institute  which  would  serve  as  a  research  hub  for  government  (AFRL  in  Mesa), 
university  (NMSU,  ASU),  and  industry  (US  Positioning  affiliates).  Dee  Andrews  of 
AFRL  was  present  for  this  meeting. 

Army  Research  Lab 

The  NMSU  Department  of  Psychology  has  been  involved  in  a  large  ARL-sponsored 
consortium  for  advanced  decision  making  technologies.  The  CERTT  Lab  has 
participated  in  this  effort  in  several  ways:  1)  identifying  tasks  or  scenarios  that  can  be 
studies  in  a  distributed  way  across  the  consortium  and  2)  sharing  event  log  data  for  a 
project  focused  on  analyzing  sequential  behavior.  In  October  of  2001  the  CERTT  Lab 
was  demonstrated  to  Mike  Strub,  Linda  Pierce,  Laurel  Allender,  and  Larry  Shattuck 
during  a  site  visit.  Other  contacts  regarding  Army  UAV  concerns  have  also  been  made 
through  Mike  Barnes  at  Fort  Huachuca,  Jay  Shively  at  NASA  Ames,  and  Lila  Laux  at 
MicroAnalysis  and  Design. 

Office  of  Naval  Research 

Nancy  Cooke  is  also  involved  in  an  ONR-supported  effort  (Susan  Chipman)  with  Peter 
Foltz.  This  effort  focuses  on  automating  the  analysis  of  team  communication  data.  The 
three  year  grant  ends  in  March  2003. 

Army  Research  Institute 

Nancy  Cooke  is  serving  in  an  advisory  capacity  to  Adrienne  Lee,  PI  for  an  Army 
Research  Institute  grant  to  explore  the  transfer  of  distributed  or  co-located  training  to 
distributed  or  co-located  mission  environments.  The  first  experiment  for  this  grant  is 
being  conducted  this  year  (summer  and  fall  2002). 

UCSD 

Nancy  Cooke  serves  as  a  consultant  to  various  VA  grants  of  Matt  Weinger,  a  UCSD 
anesthesiologist.  This  work  concerns  anesthesiology  expertise  and  teams  in  the  operating 
room. 

DIA/NTSB 

Nancy  Cooke,  Janie  DeJoode,  and  Steve  Shope  have  recently  (August  28)  observed  a 
mass  disaster  simulation  at  Denver  International  Airport  at  the  request  of  the  NTSB  (Jim 
Strusacker).  Command  and  control  centers  were  observed  and  observations  and 
recommendations  are  forthcoming. 
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Transitions 

None  to  date,  but  we  anticipate  a  number  of  opportunities  for  immediate  technology 
transition  through  the  new  research  institute  and  its  close  affiliation  with  US  Positioning. 


INVENTIONS 


None 
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Nancy  Cooke  elected  Fellow  of  the  Human  Factors  and  Ergonomics  Society  (2000) 
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